feat(vortex-row): row-oriented byte encoder (size + encode passes) by joseph-isaacs · Pull Request #8253 · vortex-data/vortex

joseph-isaacs · 2026-06-04T14:28:46Z

Summary

Adds vortex-row, a new crate that encodes one or more columnar Vortex arrays into a single
ListView<u8> whose per-row byte slices are lexicographically comparable. The byte order
matches tuple ordering of the input values under per-column sort options, so the output works
directly as a sort key / row key — the Vortex analogue of arrow-row.

This PR includes the base row-encoding API, the two scalar-function passes, the byte codec,
focused tests, a row_encode benchmark, and the row byte-layout documentation. The crate is
marked publish = false, so no public-api.lock is tracked while the API is still settling.

FSST-specific row-encoding experiments are intentionally not included in this PR.

Design

Encoding runs as two scalar functions wired behind the RowEncoder API:

Size pass — RowSize. Walks the N input columns once, classifies each column as
fixed- or variable-width, accumulates the fixed-width prefix per row, and lazily collects
per-row variable lengths. Returns Struct { fixed: u32, var: u32 } so callers read per-row
widths without materializing the constant fixed slot as a per-row buffer.
Encode pass — RowEncode. Uses those sizes to compute totals, allocate one contiguous
elements buffer, build per-row absolute offsets, then writes each column left-to-right into
its per-row slot via a write cursor that doubles as the ListView sizes array, so no
separate finalize step is needed.

The converter is effectively 2 passes for the pure-fixed-width case and 3 when
variable-length columns require the prefix-sum offsets pass.

Per-column ordering is controlled by RowSortField { descending, nulls_first }: descending
reverses the encoded value bytes, and leading sentinel bytes place nulls before or after
non-nulls independently of sort direction.

API Layout

convert_columns(cols: &[ArrayRef], fields: &[RowSortField], ctx) -> VortexResult<ListViewArray>
is the one-shot entry point; RowEncoder is the reusable form.

Item	File
`RowEncoder`, `convert_columns(_with_options)`, `compute_row_sizes(_with_options)`	`src/encoder.rs`
`RowEncode` scalar fn + encode driver	`src/encode.rs`
`RowSize` scalar fn + size/classify pass (`compute_sizes`)	`src/size.rs`
`RowEncodingOptions`, `RowSortField`	`src/options.rs`
per-dtype byte codec (`field_size` / `field_encode`)	`src/codec.rs`
`initialize(session)` + re-exports	`src/lib.rs`

Type Coverage

Supported: nulls, booleans, integer/float primitives, decimals up to 128 bits, UTF-8 and
binary, structs, and fixed-size lists.

Rejected: extension types, variant, union, and variable-size list arrays. Decimal256 is also
not implemented. Temporal extensions could be added later by normalizing them to storage arrays
at the RowEncoder boundary once the temporal ordering contract is made explicit.

Docs

Adds docs/specs/row-encoding.md, a FoundationDB-tuple-style byte-sort specification with:

sentinel summary table
BE(value) definition and examples
per-type encoding rules
unsupported-type table
worked row example at the end

The spec and vortex-row/README.md both mark the byte layout as experimental.

Testing

cargo test -p vortex-row — 23 tests passed.
uv run --all-packages make -C docs doctest — 268 doctests passed.
cargo +nightly fmt --all — clean.
cargo clippy -p vortex-row --all-targets --all-features -- -D warnings — clean.
git diff --check — clean.

Adds `vortex-row`, which encodes N columnar arrays into a single byte-comparable `ListView<u8>` (the Vortex analogue of arrow-row) for use as sort/row keys. Encoding runs as two scalar functions behind the `RowEncoder` API: a `RowSize` sizing/classification pass and a `RowEncode` pass that allocates one contiguous buffer and writes each column left-to-right into its per-row slot. Per-column ordering (`RowSortField`) controls ascending/ descending and null placement. Includes the byte codec for fixed-width, varlen, and nested canonical types, the `convert_columns`/`compute_row_sizes` helpers, round-trip + invariant tests, and arrow-row-baselined throughput benches. The crate is marked `publish = false` for now, so no public-api.lock is tracked. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

codspeed-hq · 2026-06-04T14:43:42Z

Merging this PR will improve performance by 15.37%

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 1503 untouched benchmarks
🆕 6 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Simulation	`baseline_lt[16, 65536]`	219.4 µs	247 µs	-11.17%
⚡	Simulation	`chunked_bool_canonical_into[(1000, 10)]`	46.6 µs	31.7 µs	+46.97%
⚡	Simulation	`chunked_varbinview_into_canonical[(1000, 10)]`	213.2 µs	177.1 µs	+20.39%
⚡	Simulation	`chunked_varbinview_canonical_into[(100, 100)]`	309.6 µs	274.7 µs	+12.71%
🆕	Simulation	`primitive_i64_arrow_row`	N/A	2.4 ms	N/A
🆕	Simulation	`primitive_i64_vortex`	N/A	1.5 ms	N/A
🆕	Simulation	`struct_mixed_arrow_row`	N/A	18.7 ms	N/A
🆕	Simulation	`struct_mixed_vortex`	N/A	22.9 ms	N/A
🆕	Simulation	`utf8_arrow_row`	N/A	8.6 ms	N/A
🆕	Simulation	`utf8_vortex`	N/A	9.3 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing claude/nice-archimedes-yjGyO (0892a82) with develop (f127357)}

Add a CodSpeed shard for `vortex-row` so the `row_encode` divan benchmarks (vortex vs arrow-row) build and run in CI alongside the other crates. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The row encoder builds the output `(elements, offsets, sizes)` triple itself, so the invariants `ListViewArray::try_new` validates (monotone offsets, per-row slices within bounds and disjoint) already hold by construction. Skip the revalidation walk via `new_unchecked`. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Introduce `ValidityKind`/`resolve_validity`: resolve a column's validity once, materializing the per-row mask only when the column may actually contain nulls. The size pass for varbinview and the bool and primitive encoders now branch once on validity, so the all-valid path drops the per-row `mask.value(i)` check (and mask allocation) entirely. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Every byte of the output range is written by some encoder: fixed-width null rows write sentinel + explicit zero-fill, varlen encoders zero-pad their final partial block, and struct/FSL null parent bodies are overwritten with the canonical null encoding. The pre-zero-init memset is therefore redundant, so replace it with `set_len`, saving a `total_len`-byte memset per call. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Materialize the listview offsets buffer with `set_len` + a slice write instead of per-row `push`. For the pure-fixed path, `iter_mut().enumerate()` lets LLVM auto-vectorize `offsets[i] = i * fixed_per_row` (no per-element bounds or capacity checks). `nrows` is validated to fit u32 at function entry, so the cast is exact. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Write the mixed (fixed + varlen) offsets through `iter_mut().zip` with wrapping arithmetic, mirroring the pure-fixed path: this elides per-element bounds checks so the `i * fixed_per_row` multiply auto-vectorizes while the varlen prefix sum stays a cheap sequential accumulator. The total is validated to fit u32 upstream, so the wrapping operations never actually wrap. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

…ping The varlen body writer was a per-byte XOR loop. Split it into an ascending fast path (`copy_nonoverlapping` of each 32-byte block plus a single stamped continuation byte, then a partial final block) and a descending path that XORs a u64 at a time via `xor_copy_block` for a vectorizable inner loop. The emitted bytes are identical to the previous implementation for every length and direction (full-block counts and final length byte match exactly); only the write strategy changes. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Replace the `with_iterator` traversal in `encode_varbinview` with a direct walk over the view array: cache the data-buffer slices once, then for each row read the bytes straight from the inlined view slot or the referenced buffer at `offset..offset+len`. This drops the iterator's per-row option/bounds machinery. Validity is resolved once via `resolve_validity`, keeping the no-nulls path branch-free on validity. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

The auto-vectorized offset loops and the varlen block writer used raw `as` casts that trip this crate's `cast_possible_truncation` lint. Iterate a `u32` counter instead of casting `usize` per element, and use `u8`/`u32` `try_from` for the varlen final-block length byte and total byte count. No behavior change. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Classify each column in the size pass (`ColKind` + `first_varlen_idx`): a fixed-width column with no varlen column before it has a constant within-row offset, so its write position is pure arithmetic (`i * fixed_per_row + prefix + var_prefix[i]`) with no per-row cursor. Route those columns through `field_encode_fixed_arithmetic`; the cursor path is seeded to start at the first varlen column. Primitive columns in the pure-fixed case use a `chunks_exact_mut` hot loop (matching arrow-row's not-null path); all other fixed types reuse the cursor encoder at the computed offsets, so output is byte-identical. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Run the vortex-row row_encode benchmarks as part of the existing 'Storage formats' shard rather than adding a dedicated ninth shard. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

FSST is not order-preserving, so row keys must be the decompressed bytes; the only strategy today is decompress to a canonical VarBinView then row-encode it. This bench measures that path and its two phases (decompress-only, and row-encode of an already-decompressed column) on compressible multi-block strings, to quantify the opportunity for a future fused FSST row-encode kernel: the phases are additive (decompress ~46%, row-encode ~54%), and the row-encode phase re-reads/re-writes the decompressed bytes a fused kernel could emit once. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Apply nightly rustfmt formatting to the FSST benchmark added in the previous commit. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Adds `fsst_fast_fused`: bulk-decompresses the FSST code heap straight into a contiguous buffer (no intermediate VarBinViewArray) and block-encodes rows directly into the row-key ListView using the stored uncompressed_lengths (free size pass), with the same no-zero-init / no-extra-copy techniques as the row encoder. Lets us compare the fused path head-to-head against decode-then-convert. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Adds `fsst_fast_scatter`: keeps FSST's fast contiguous bulk decompressor but runs it into a cache-resident scratch one row-batch at a time, scattering each row into block form from cache so the decompressed bytes never round-trip through main memory. A one-time assert_arrays_eq! check confirms it produces byte-identical row keys to the straightforward fused path. Result: fast_scatter is on par with fast_fused (no speedup) — the decompressed buffer is already consumed cache-warm in the simple fused path, so avoiding the round-trip saves nothing; the workload is CPU-bound on FSST symbol decode plus block-copy. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

dimitarvdimitrov

SGTM

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

AdamGS · 2026-06-05T14:20:28Z

+fn add_size_const(sizes: &mut [u32], add: u32) {
+    for s in sizes.iter_mut() {
+        *s += add;
+    }
+}
+
+fn add_size_null(arr: &NullArray, sizes: &mut [u32]) {
+    debug_assert_eq!(arr.len(), sizes.len());
+    // Just a sentinel byte per row.
+    for s in sizes.iter_mut() {
+        *s += 1;
+    }
+}
+
+fn add_size_primitive(arr: &PrimitiveArray, sizes: &mut [u32]) {
+    let width = byte_width_u32(arr.ptype().byte_width());
+    add_size_const(sizes, encoded_size_for_fixed(width));
+}
+
+fn add_size_decimal(arr: &DecimalArray, sizes: &mut [u32]) {
+    let width = byte_width_u32(arr.values_type().byte_width());
+    add_size_const(sizes, encoded_size_for_fixed(width));
+}


all these are just one-liners - sizes.iter_mut().for_each(...)

AdamGS · 2026-06-05T14:22:43Z

+                let pos = (row_offsets[i] + col_offset[i]) as usize;
+                out[pos] = non_null;
+                // false=0x01, true=0x02 so false < true; XOR for descending
+                let raw = if bits.value(i) { 0x02u8 } else { 0x01u8 };


as u8 + 1?

AdamGS · 2026-06-05T14:23:04Z

+                let pos = (row_offsets[i] + col_offset[i]) as usize;
+                if mask.value(i) {
+                    out[pos] = non_null;
+                    let raw = if bits.value(i) { 0x02u8 } else { 0x01u8 };


same as above

AdamGS · 2026-06-05T14:25:06Z

+    /// Returns the sentinel byte to write for a non-null value.
+    #[inline]
+    pub(crate) fn non_null_sentinel(&self) -> u8 {
+        // Non-null is always 0x01. Null choices are < or > 0x01.
+        0x01
+    }
+
+    /// Returns the sentinel byte to write for a null value.
+    #[inline]
+    pub(crate) fn null_sentinel(&self) -> u8 {
+        if self.nulls_first {
+            // Nulls before non-nulls (smaller byte sorts first).
+            0x00
+        } else {
+            // Nulls after non-nulls (larger byte sorts later).
+            0x02
+        }
+    }


These (non_null_sentinel, null_sentinel) shouldn't be part of RowSortField, they don't really fit the rest of the pattern

AdamGS · 2026-06-05T14:25:33Z

+    }
+}
+
+const FIELDS_INLINE: usize = 4;


explain? not sure I get what's going on here

AdamGS · 2026-06-05T14:25:53Z

+/// reverses the encoded value bytes for that column. Null placement is controlled separately,
+/// so nulls keep the requested position relative to non-null values in either direction.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+pub struct RowSortField {


nit - RowSortFieldOptions?

AdamGS · 2026-06-05T14:31:30Z

@@ -0,0 +1,615 @@
+// SPDX-License-Identifier: Apache-2.0


I think there are two types of tests that are missing here:

Lock in some reference values (Like in the docs page).

some property testing - generate a dtype, generate values for that dtype and assert their order doesn't change.

AdamGS · 2026-06-05T14:34:14Z

@@ -0,0 +1,539 @@
+# Row Encoding Byte Sort Specification


can more of this content be in the lib.rs? I think that's a much more accessible location, and more helpful for reading the code.

AdamGS · 2026-06-05T14:37:04Z

@@ -0,0 +1,539 @@
+# Row Encoding Byte Sort Specification


How much trust do you have in this scheme?

AdamGS · 2026-06-05T14:38:56Z

@@ -0,0 +1,1257 @@
+// SPDX-License-Identifier: Apache-2.0


This file is almost half this PR, probably worth splitting it into a module with multiple files

AdamGS · 2026-06-05T14:41:39Z

+/// Encoders pattern-match on this once before their inner loop so the no-nulls fast path
+/// avoids per-row `mask.value(i)` branches entirely, and the nullable path materializes the
+/// mask exactly once.
+pub(crate) enum ValidityKind {


Isn't this just AllOr from vortex-mask?

AdamGS · 2026-06-05T14:42:10Z

+    1 + value_bytes
+}
+
+fn byte_width_u32(width: usize) -> u32 {


byte width always fit in a usize, why can't we just cast?

AdamGS · 2026-06-05T14:43:18Z

+/// path (no varlen before this column, so the within-row position is constant per row) and
+/// the cursor-write path.
+#[derive(Clone, Copy, Debug)]
+pub(crate) enum ColKind {


AdamGS · 2026-06-05T14:46:14Z

+            // Each row has `list_size` fixed-width elements regardless of null parent mask.
+            let body = w
+                .checked_mul(u32::try_from(list_size).vortex_expect("list_size fits u32"))
+                .vortex_expect("FSL body width overflow");


AdamGS · 2026-06-05T14:49:13Z

+            ptype.byte_width(),
+        )))),
+        DType::Decimal(dt, _) => {
+            let vt = DecimalType::smallest_decimal_value_type(dt);


is shrinking the decimal here sound? I think it makes the behavior much less predictable.

AdamGS · 2026-06-05T14:49:51Z

+                match row_width_for_dtype(&field_dtype)? {
+                    RowWidth::Fixed(w) => {
+                        total = total.checked_add(w).ok_or_else(|| {
+                            vortex_error::vortex_err!("Struct row width overflows u32")


AdamGS · 2026-06-05T14:52:07Z

This PR is pretty big, I've made an effort but given that its already merged I have other priorities.

Follow-up to the PR #8253 review pass: - Make the size pass fully fallible: add_size_* now return VortexResult and use checked arithmetic, so an input whose per-row encoding exceeds u32::MAX surfaces a VortexError instead of panicking via vortex_expect. encoded_size_for_non_empty_varlen and encode_non_empty_varlen_body likewise return VortexResult for their byte-total overflow checks. - Drop the #[allow(cast_possible_truncation)] on byte_width_u32; use u32::try_from with an infallible-invariant expect instead of a bare cast. - Add reference_row_bytes_match_spec: encodes the worked-example row from docs/specs/row-encoding.md and asserts the exact encoded bytes, pinning the byte layout and keeping the spec honest. Signed-off-by: Claude <noreply@anthropic.com> https://claude.ai/code/session_019GXtsg21qhpxDVD9ZUpFTx

joseph-isaacs added the changelog/feature A new feature label Jun 4, 2026 — with Claude

ci(vortex-row): run row_encode benchmarks on CodSpeed

083c7f3

Add a CodSpeed shard for `vortex-row` so the `row_encode` divan benchmarks (vortex vs arrow-row) build and run in CI alongside the other crates. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs marked this pull request as ready for review June 4, 2026 15:35

joseph-isaacs requested a review from robert3005 June 4, 2026 15:41

joseph-isaacs added 10 commits June 4, 2026 17:24

ci(vortex-row): fold row_encode benchmarks into CodSpeed shard 8

2fc07fa

Run the vortex-row row_encode benchmarks as part of the existing 'Storage formats' shard rather than adding a dedicated ninth shard. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs force-pushed the claude/nice-archimedes-yjGyO branch from 81de8fa to 2fc07fa Compare June 5, 2026 08:59

joseph-isaacs added 6 commits June 5, 2026 11:33

vortex-row: rustfmt the fsst row-encode benchmark

d6f1f4e

Apply nightly rustfmt formatting to the FSST benchmark added in the previous commit. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

fix

b3411f1

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

fix

a213cdd

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

dimitarvdimitrov reviewed Jun 5, 2026

View reviewed changes

dimitarvdimitrov approved these changes Jun 5, 2026

View reviewed changes

joseph-isaacs added 2 commits June 5, 2026 14:17

reduce coverage job disk pressure

2a3ae8f

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

fix

0892a82

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs merged commit 7482863 into develop Jun 5, 2026
68 of 70 checks passed

joseph-isaacs deleted the claude/nice-archimedes-yjGyO branch June 5, 2026 14:09

AdamGS reviewed Jun 5, 2026

View reviewed changes

Conversation

joseph-isaacs commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Design

API Layout

Type Coverage

Docs

Testing

Uh oh!

codspeed-hq Bot commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 15.37%

Performance Changes

Uh oh!

dimitarvdimitrov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdamGS Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdamGS Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdamGS commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joseph-isaacs commented Jun 4, 2026 •

edited

Loading

codspeed-hq Bot commented Jun 4, 2026 •

edited

Loading

AdamGS Jun 5, 2026 •

edited

Loading

AdamGS Jun 5, 2026 •

edited

Loading